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There is widespread interest in identifying computational and neurobiological mechanisms 
that influence the ability to choose long-term benefits over more proximal and readily 
available rewards in domains such as dietary and economic choice. We present the results 
of a human fMRI study that examines how neural activity relates to observed individual 
differences in the discounting of future rewards during an intertemporal monetary choice 
task. We found that a region of left dorsolateral prefrontal cortex (dIPFC) BA-46 was 
more active in trials where subjects chose delayed rewards, after controlling for the 
subjective value of those rewards. We also found that the connectivity from dIPFC 
BA-46 to a region of ventromedial prefrontal cortex (vmPFC) widely associated with the 
computation of stimulus values, increased at the time of choice, and especially during trials 
in which subjects chose delayed rewards. Finally, we found that estimates of effective 
connectivity between these two regions played a critical role in predicting out-of-sample, 
between-subject differences in discount rates. Together with previous findings in dietary 
choice, these results suggest that a common set of computational and neurobiological 
mechanisms facilitate choices in favor of long-term reward in both settings. 

Keywords: fMRI, temporal discounting, vmPFC, DLPFC, effective connectivity, individual differences, delay of 
gratification 



INTRODUCTION 

Impaired ability to delay gratification is thought to play a criti- 
cal role in sub-optimal decision-making, and in conditions like 
addiction and obesity (Chambers et al., 2007; Monterosso and 
Ainslie, 2007; Peters and Buchel, 2011). As a result, there is a 
widespread, on-going effort to characterize the computational 
and neurobiological mechanisms underlying this form of self- 
control. Two types of paradigms have been widely used in behav- 
ioral neuroscience to examine these mechanisms. First, are tasks 
involving intertemporal decisions between rewards, often money, 
in which subjects choose between sooner-smaller amounts and 
later-larger ones (Rachlin, 2000; Ainslie, 2001; McClure et al., 
2004; Kable and Glimcher, 2007; McClure et al, 2007; Ballard and 
Knutson, 2009; Gregorios-Pippas et al, 2009; Carter et al., 2010; 
Monterosso and Luo, 2010; Peters and Buchel, 2010, 2011; Luo 
et al., 2012). Second, are tasks involving dietary choices, in which 
subjects make choices between foods that vary in their tastiness 
and healthiness (Hare et al, 2009, 201 la; Volkow et al., 201 1). 

In previous work investigating dietary self-control, we found 
important commonalities and differences between successful and 
unsuccessful dieters (Hare et al., 2009). Behaviorally, the two 
groups differed on the relative weight that they placed on the 
health and taste attributes of foods in making their decisions 
(with successful dieters weighting both health and taste, and 
unsuccessful dieters weighting only taste). Neurally, the ventro- 
medial prefrontal cortex (vmPFC) encoded the value of foods 
at the time of choice equally for both groups. The critical 



difference had to do with the role of left dorsolateral prefrontal 
cortex (dIPFC). In successful dieters, dIPFC came on-line and 
exhibited increased effective connectivity with vmPFC during 
choices that required self-control (e.g., refusing to eat tasty, but 
unhealthy candy). In contrast, unsuccessful dieters did not exhibit 
this pattern of connectivity. Furthermore, in a subsequent study 
we found that non-dieting participants behaved like successful 
dieters if they were given an exogenous reminder to pay attention 
to health information, and that the reminder activated the same 
dlPFC-vmPFC networks that successful dieters activated on their 
own (HareetaL, 2011a). 

These findings led us to propose the following model of the 
computational and neurobiological processes at work in self- 
control (Hare et al, 2009, 2011a,b; Rangel and Hare, 2010; Harris 
et al., 2013). In the model, the vmPFC computes the value of 
options at the time of decision, by first assessing their vari- 
ous attributes, and then integrating them into a net value for 
the option as a whole. Importantly, "basic" attributes like tasti- 
ness might always be represented in the final value. However, 
more abstract attributes like healthiness are only represented, 
or are represented more strongly, if the dIPFC comes online 
and modulates activity in vmPFC so that its value computa- 
tions incorporate them. This modulation is critical for optimal 
decision-making because, if some of the attributes are not rep- 
resented or weighted properly, the vmPFC will assign values to 
options that are not consistent with the long-term, goal-relevant 
(e.g., proper nutrition) rewards that they generate. 
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An important open question is whether this model is also at 
work in other decision domains, such as those involving intertem- 
poral monetary tradeoffs. This question is important because 
comparing the mechanisms at work in different decision contexts 
is a critical step in identifying common mechanisms that facilitate 
self-control. Theoretically, these circuits should also influence the 
degree of discounting for delayed rewards in the case of intertem- 
poral choice, as long as dlPFC modulation of vmPFC can lead to 
an increased (or decreased) weighting for delayed rewards. 

Here we address this open question by testing the following 
three hypotheses. First, we hypothesized that regions of left dlPFC 
similar to those that are more active during self-control in dietary 
choice would also be more active in intertemporal choice when 
the subjects choose the larger-delayed payment over the money 
available today, after controlling for their relative subjective val- 
ues. Note that it is crucial to control for the subjective values 
because, if the subjective value of the delayed reward is large 
enough, the decision to wait becomes trivial. Second, we hypothe- 
sized that effective connectivity from left dlPFC to vmPFC would 
be stronger during trials in which subjects choose larger-delayed 
rewards (again controlling for subjective value), which is consis- 
tent with the idea that dlPFC can modulate the value signals in 
vmPFC so that they place more weight on the value of delayed 
payouts. Third, we hypothesized that the levels of activation in 
dlPFC, as well as its effective connectivity to vmPFC, would help 
to explain differences in discount rates across subjects. 

These hypotheses are based not only on previous work in 
dietary choice, but also on findings from the previous literature 
on goal-directed choice. First, areas of vmPFC have consistently 
been shown to correlate with stimulus values at the time of choice 
across a wide variety of decision contexts (Tom et al, 2007; 
Boorman et al, 2009; Lebreton et al, 2009; Basten et al, 2010; 
Hare et al, 2010; Plassmann et al, 2010; Shenhav and Greene, 
2010; Clithero et al., 2011; Kahnt et al, 2011; Park et al, 2011), 
including decisions involving intertemporal tradeoffs (Kable and 
Glimcher, 2007; Ballard and Knutson, 2009; Hare et al, 2009, 
201 la; Carter et al., 2010; Peters and Buchel, 2010). Second, previ- 
ous studies have associated responses in left dlPFC with choosing 
to wait for delayed monetary rewards using transcranial magnetic 
stimulation (TMS) and fMRI (McClure et al, 2004; Figner et al., 
2010; Luo et al., 2012). In particular, Figner et al. (2010) showed 
that temporarily reducing activity in left dlPFC via TMS results 
in subjects making more impatient choices, thus, establishing a 
causal role for this region in temporal discounting. Third, recent 
studies have found that resting-state connectivity in networks 
including left dlPFC was correlated with discount rates (Gianotti 
etal, 2012; Li etal, 2013). 

Despite the attractiveness of the theory, and the body of consis- 
tent evidence, critical questions remain open. In particular, none 
of the previous studies have examined the effective connectiv- 
ity between dlPFC and vmPFC during intertemporal monetary 
choices, nor have they established that the dlPFC influences dis- 
count rates through a mechanism that involves the modulation 
of the stimulus values computed in vmPFC, or that the effective 
connectivity runs from dlPFC to vmPFC, and not the other way 
around. Here we are able to address these questions by estimat- 
ing Dynamic Causal Models (Friston et al., 2003), and using those 



parameter estimates to explain and predict differences in discount 
rates across individuals. 

MATERIALS AND METHODS 
PARTICIPANTS 

Twenty-seven subjects (18 males; age: mean = 24.1 years; range = 
19-40) were included in the study. Two additional subjects were 
excluded because of excessive head motion during the scanning 
session (>2mm in translation or rotation). All participants had 
normal or corrected-to-normal vision, no history of neurological, 
psychiatric, or metabolic illness, and were not taking any medica- 
tions that interfere with the blood oxygenation level-dependent 
(BOLD) signal at the time of scanning. The Institutional Review 
Board at California Institute of Technology approved the methods 
and procedures used in this study. 

INTERTEMPORAL CHOICE (ITC) TASK 

On every trial, subjects chose between getting $25 at the end of 
the experiment, or getting an equal or larger amount at a later 
date. The later offers ranged from $25 to $54; with a delay from 7 
to 200 days. Subjects made 216 decisions. The unique combina- 
tions of amount and delay used are shown in Table 1. All subjects 
saw the same set of options, although in different random orders. 
Each option was shown twice. Note that by presenting all subjects 
with the same options we were able to control for the objective 
reward levels when testing how neural activity relates to discount 
rates. Although beneficial for the hypotheses tested in previous 
studies (Kable and Glimcher, 2007, 2010; Peters and Buchel, 2009, 
2010), tailoring the choice sets around the indifference points of 



Table 1 | Amounts by delay. 



Delay Amount 
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25 


26 


28 


30 


32 


35 


10 


25 


26 


27 


29 


30 


32 


12 


25 


26 


28 


31 


33 


35 


14 


25 


26 


28 


32 


35 


39 


21 


26 


27 


29 


30 


32 


38 


25 


27 


29 


31 


33 


35 


46 


28 


26 


28 


32 


35 


39 


46 


30 


26 


27 


29 


30 


32 


38 


40 


27 


33 


35 


40 


47 


54 


45 


26 


29 


31 


35 


40 


46 


50 


27 


30 


35 


40 


46 


54 


60 


29 


33 


35 


40 


47 


54 


90 


26 


30 


33 


40 


46 


54 


95 


31 


33 


35 


40 


47 


54 


100 


26 


31 


38 


39 


46 


54 


150 


31 


33 


35 


40 


47 


54 


180 


27 


31 


35 


39 


46 


54 


200 


26 


28 


35 


39 


47 


54 



Delays are listed in days and amounts in USD. The combinations of amount and 
delay were chosen to facilitate the estimation of hyperbolic discounting param- 
eters within the range 0.0005-0.05 that is commonly reported in monetary 
discounting tasks. 
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each subject would create a confound with objective value when 
examining how individual differences in neural activity relate 
to discount rates because less patient subjects would be shown 
delayed rewards with higher monetary values. 

As described in Figure 1A, each trial began with an offer pre- 
sented onscreen. Participants were required to press within 3 s to 
indicate whether or not they accepted the delayed reward offered. 
Only the varying delayed option was presented onscreen. A but- 
ton press response resulted in the termination of the offer screen, 
and the appearance of a feedback screen for 250 ms displaying 
"Yes," if the delayed offer was accepted, or "No," if it was rejected. 
The phrase "No decision received" was displayed if the subject 
failed to respond within 3 s (mean = 2% of trials, standard devi- 
ation 5%, median = 0%). Trials were separated by a fixation 
cross of random duration (uniform: 2-6 s). The assignment of 
left/right button presses to accept/reject responses was counter- 
balanced across subjects. At the end of the experiment a single 
trial was randomly chosen and implemented: subjects received 
the chosen option in addition to $50 (available immediately) for 
participating in the study. All payments were made using prepaid 
debit cards given to the subjects at the end of the experiment. 
This allowed us to make the delayed payments available on the 
appropriate date, without requiring subjects to return to the lab. 

BEHAVIORAL DATA ANALYSIS 

We estimated an individual discount factor (denoted by k) 
for each subject using maximum likelihood. In particular, we 
assumed that subjects assigned value to the delayed options using 
a hyperbolic discounting function, in which the value of $A with 



$25 today or: 
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FIGURE 1 | Task design and behavioral data. (A) Example display 
screens and timing parameters. (B) Choice curve displaying the probability 
of choosing the larger, delayed reward. The y-axis shows the probability of 
selecting the future reward and the x-axis displays the stimulus value of the 
future reward. Error bars represent the standard error of the mean. (C) Bar 
graph showing the distribution of discounting parameters across subjects. 
The x-axis represents individual subjects and the y-axis is the magnitude of 
the discount parameter k from a hyperbolic discounting function. 



a delay of D days is given by 

dSV = A/(l + kD), 

where dSV denotes the discounted stimulus value. We also 
assumed that the probability of accepting the delayed option is 
given by the soft-max function 

P(Yes) = (1 + exp(b*(25 - dSV))y\ 

where b is a non-negative parameter that modulates the slope of 
the psychometric choice function. Note that in this formula the 
value of the constant reference option is $25. 

IMAGING DATA ACQUISITION 

FMRI data were collected in a Siemens (Erlangen, Germany) 3.0 
Tesla Trio MRI scanner. Using an eight-channel, phased array 
head coil, we collected gradient echo T2* -weighted echoplanar 
(EPI) images with BOLD contrast. In order to optimize BOLD 
sensitivity, we used a tilted acquisition in an orientation 30° 
oblique to the anterior-posterior commissure line (Deichmann 
et al., 2003). The imaging parameters were as follows: TR = 
2500ms; TE = 30ms; flip angle = 80°; FOV = 192mm; in- 
plane resolution = 3 x 3-mm; and 40 3-mm slices (0.3-mm 
gap) with ascending acquisition. While in the scanner, sub- 
jects completed two runs of the ITC task (with 323 volumes 
acquired per run). They also completed an additional task involv- 
ing the degustation of liquid rewards that is not relevant to this 
study (task order was counterbalanced across participants). High- 
resolution, whole-brain Tl-weighted structural images {TR = 
1500 ms; TE = 3.05 ms; flip angle = 10°; voxel resolution = 
1 mm 3 ; single-shot, ascending acquisition) were also collected for 
each participant. These images were co-registered with the their 
respective EPI images to assist with the anatomical localization of 
the functional activations. 

FMRI DATA PREPROCESSING 

Imaging data were preprocessed using SPM8 (Wellcome 
Department of Imaging Neuroscience, Institute of Neurology, 
London, UK). Data were corrected for motion with realign- 
ment to the mean image, spatially normalized to the Montreal 
Neurological Institute EPI template, resampled to 3 mm 3 voxels, 
and spatially smoothed using a Gaussian kernel (full-width- 
at-half-maximum = 8 mm). Data were also temporally filtered 
using a filter width of 128 s. 

GLMs 

We estimated two different mixed effect models of the BOLD 
responses, with first order auto correlation correction [AR(1)]. 
The models were designed to localize in our sample the areas of 
vmPFC that, as discussed in the Introduction, have been repeat- 
edly shown to correlate with stimulus values at the time of choice. 
The models are identical except for the specification of the value 
modulators. 

The first model, GLM-dSV, had the following regressors of 
interest: (1) An indicator function beginning at the onset of each 
decision screen with duration equal to the reaction time for that 
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trial, (2) the indicator function modulated by the subject spe- 
cific value of each delayed offer (dSV), and (3) the indicator 
function modulated by the variable Accept (which equals 1 if the 
subject chooses the delayed outcome, and zero otherwise). The 
third regressor was orthogonalized with respect to the second one 
in order to assign any shared variance between them to the dSV 
regressor. The model also included session dummies, linear time 
trends, and head movements as regressors of no interest. 

The second model, GLM-rdSV, was identical except for the 
specification of the parametric regressor. In particular, for the 
reasons described in the Results section, we defined a relative 
discounted subjective value (rdSV) variable, which is equal to 
dSV —25 for subjects that choose the delayed variable more than 
50% of the time (15 subjects), and is equal to 25-dSV for those 
that choose the immediate option more frequently (12 subjects). 

Both GLMs were estimated in three steps. First we estimated 
the model at the individual level. Second, we calculated the fol- 
lowing first-level single-subject contrasts: regressor 2 (dSV or 
rdSV) vs. baseline, and regressor 3 (Accept) vs. baseline. Third, we 
calculated second-level group contrasts using one-sample t"-tests 
on the single-subject contrasts. 

We controlled for multiple comparisons at the cluster level 
using an individual voxel threshold of p < 0.005 to achieve a 
whole brain corrected (WBC) ^-values less than 0.05 (cluster sizes 
are listed in each table). We also used small volume corrections 
(SVCs) in areas of a priori interest to the study of the self- 
control mechanisms that are at the core of the hypotheses tested 
here. We carried out an SVC in the vmPFC using an anatomical 
mask based on the AAL atlas (Tzourio-Mazoyer et al., 2002) that 
included the rectal gyrus, medial orbitofrontal, and anterior cin- 
gulate cortex below z = 5 (1619 3-mm 3 voxels). A region in left 
dorsolateral PFC, in Brodmann Area (BA) 46, has been shown to 
play a role in various types of self-control tasks (Hare et al., 2009, 
2011a; Figner et al, 2010). Because anatomical masks of dlPFC 
lacked the required specificity to isolate this region, we carried 
out SVC using a mask composed of a 10-mm radius sphere cen- 
tered around the target coordinates (x, y, z = —36, 30, 27) used 
by Figner et al. (2010) to demonstrate a causal role of dlPFC 
on discounting behavior. The TMS stimulation of left dlPFC in 
this study was directed to the F3 location of the international 
10-20 system for electrode placement, and we used the trans- 
formation algorithms in the Mtinster T2T-Converter software 
(http://wwwneuro03.uni-muenster.de/ger/t2tconv/) to compute 
an estimate of the underlying MNI coordinates. 

To assess the impact of the dSV regressor on model fit within 
the reward valuation network identified in two recent meta- 
analyses (Bartra et al., 2013; Clithero and Rangel, 2013), we re- 
estimated GLM-dSV and a reduced form of GLM-dSV using the 
Variational Bayes routines in SPM8 (Penny et al, 2003, 2005). The 
reduced form of GLM-dSV excluded the parametric regressor for 
dSV, but included all other regressors described above. Following 
estimation of both models at the single subject level, we used a 
random effects Bayesian model selection (BMS) procedure (Rosa 
et al., 2010) to assess the variance explained by the dSV regressor 
independent of the sign (+/— ) on its coefficient in a fashion simi- 
lar to an F-test, but additionally accounting for model complexity. 
This BMS procedure generated exceedance probabilities from the 



model evidence for each GLM. The exceedance probabilities cor- 
respond to the belief that the full version of GLM-dSV is more 
likely than the reduced version given the data from all subjects 
(or vice versa). We evaluated the exceedance probability in all 
voxels within the conjunction of masks generated from recent 
meta-analyses on reward valuation. Specifically, we formed the 
conjunction mask from the voxels shown in Clithero and Rangel's 
(2013) Figure 3 and Bartra et al. (2013) Figure 3A. This mask 
included voxels in vmPFC, ventral striatum (vStr), and posterior 
cingulate cortex (PCC) consistently found to positively corre- 
late with the value of reward across reward types and decision 
contexts in the meta-analyses listed above. This mask can be 
downloaded from the following website: http://www.rnl. caltech. 
edu/resources/index.html. 

DYNAMIC CAUSAL MODELING (DCM) 

We tested the hypothesis that the effective connectivity from left 
dlPFC-BA46 to vmPFC plays a critical role in self-control using 
DCM (Friston et al., 2003). The analysis proceeded in several 
steps. 

First, for each subject we extracted average activation time 
courses from vmPFC and left dlPFC-BA46. In particular, for every 
subject we defined an ROI with a 5-mm radius, and a center given 
by each subject's most significant voxel within the group ROIs. 
The group ROI in vmPFC was defined based on the conjunc- 
tion between voxels showing an effect for the rdSV and Accept 
regressors from GLM-rdSV at an uncorrected threshold of p < 
0.005. The group ROI in dlPFC-BA 46 was defined as all voxels 
showing an effect for the Accept regressor from GLM-rdSV at an 
uncorrected threshold of p < 0.005. 

Second, we optimized the basic architecture of the DCM, in 
terms of where experimental inputs entered. To do so, we esti- 
mated 64 different DCMs that could be organized into four 
different families (Figure 3A), based on how the variables rdSV 
and Accept affect activity in dlPFC and vmPFC. Each family con- 
tained 16 models that varied in terms of the combinations of 
connectivity between vmPFC and dlPFC-BA46 as a function of 
three events: fixation, all choice periods, and choice periods in 
which the delayed option is selected. 

Third, we compared each model family using BMS (Stephan 
et al., 2009) to determine the most likely pattern of task related 
inputs into dlPFC and vmPFC. 

Fourth, having optimized the model inputs, we calculated the 
parameter estimates and posterior probabilities of the full model 
(i.e., the one containing coupling parameters from dlPFC-BA46 
to vmPFC and vice versa for all choice types and inter-trial fixa- 
tion, as shown in Figure 3). Parameter estimates were computed 
using Bayesian parameter averaging (BPA) over subjects (Kasess 
et al, 2010). For completeness, we also tested the effective connec- 
tivity parameters from dlPFC-BA46 to vmPFC using two-tailed, 
one-sample f-tests against zero across individuals. 

PREDICTION EXERCISE 

We tested the hypothesis that the effective connectivity from left 
dlPFC-BA46 to vmPFC predicts between-subjects differences in 
the discount rate using the following out-of-sample prediction 
exercise. For every subject, we estimated the following linear 



Frontiers in IMeuroscience | Decision Neuroscience 



March 2014 | Volume 8 | Article 50 | 4 



Hare et al. 



Neural systems predicting individual discounting 



regression using elastic net regularization (Zou and Hastie, 2005) 
(alpha parameter = 0.3) on the other N— 1 subjects. 

y = p 0 + + xi$2 H h Xpfip 

Where the dependent variable y was the log of the individual dis- 
count rates (k) and the explanatory variables xi , . . . , Xp were the 
complete set of estimated DCM parameters for each functional 
run (exclusive of the hemodynamic parameters) listed in Table 7. 
One advantage of the elastic net regularization is that the regres- 
sion model is more robust to correlated predictor variables such 
as the DCM parameters for separate runs from the same subject. 

We then used the fitted coefficients from the elastic net model 
to predict if the discount parameter for the excluded individual 
was above or below the mean value for the N— 1 subjects. The pro- 
cedure was repeated for every subject. Finally, we computed the 
balanced accuracy of the prediction using the confusion matrix, 
in which the rows represent the true labels and the columns rep- 
resent the predicted labels, generated by our classification results 
(Brodersen et al., 2010). Briefly, this method controls for any 
imbalance in the data classes that may bias the classifier accuracy. 
The balanced accuracy is computed as 

1 / TP TN \ 

2 VTP + FN + TN + FP / 

where TP, FN, TN, and FP represent the number of true positives, 
false negatives, true negatives, and false positives respectively. 

In order to further test the specificity of our findings, we car- 
ried out different versions of this prediction exercise, in which 
subsets of the DCM parameters were excluded (see the Results 
section for details), or other candidate regions replaced left 
dlPFC-BA46 in the DCM. In all cases, we used the same fully 
connected DCM model with a fixed input to dlPFC-BA46 (or its 
replacement, when appropriate) on accepted trials and an input 
parametrically varying with the subjective value of the delayed 
reward to vmPFC. 

SPECIFICITY TESTS 

These tests were designed to test the specificity of left BA46 activ- 
ity and connectivity on the results. To do this, we repeated the 
DCM and prediction exercises described above using the left 
dlPFC-BA9 region listed in Table 6, or an ROI created by mir- 
roring the 10 mm sphere centered on the estimated coordinates 
from Figner et al. (2010) to the right hemisphere. 

RESULTS 
CHOICE BEHAVIOR 

We began the analysis by estimating individual discount rates 
(denoted by k) using maximum likelihood, and under the well- 
validated assumption that subjects exhibit hyperbolic discounting 
(Frederick et al., 2002; Green and Myerson, 2004; McKerchar 
et al., 2009). These estimates also allowed us to compute the dis- 
counted stimulus value (dSV) that each subject assigned to each 
option. As shown in Figure IB, which depicts the group's psy- 
chometric choice curve, the estimated values provided a good 
description of the choice data. Figure 1C provides an ordered 
histogram of the individual estimates of the discount parameter k 



(with larger values denoting more frequent choices for immediate 
reward), and shows that there were sizable differences across indi- 
viduals. This is important because one of the goals of the study is 
to relate individual differences in brain activity to differences in 
discounting behavior. 

REACTION TIMES 

Subjects responded well under the time limit of 3 s for both 
immediate [mean = 1.22 s, SD = 0.24 s, £(26) = — 39.2, p< 
0.001] and delayed choices [mean = 1.20 s, SD = 0.25 s, £(26) = 
—37.2, p < 0.001]. An ANOVA on reaction times as a func- 
tion of choice (accept delayed offer vs. take money now) and 
group (those who chose to wait for delayed rewards on the 
majority of trials — wait group, WG — vs. those who most often 
took the money available today — now group, NG) showed no 
main effects of choice or group [F(i t 4g) = 0.10, p = 0.76 and 
_F(! 4g ) = 0.49, p = 0.49, respectively]. However, there was an 
interaction between the tendency to wait and choice [F(i ; 48) = 
4.79, p < 0.05]. This interaction was driven by the fact that WG 
subjects showed faster reaction times when choosing the delayed 
reward (mean = 1133 ± 70 ms) than when choosing the imme- 
diate reward [mean = 1277 ± 63 ms; t (u) = -3.99, p < 0.001], 
whereas NG subjects had slower reaction times when choosing the 
delayed reward (mean = 1278 ± 62 ms) compared to immediate 
rewards [mean= 1143 ± 60 ms; t(n) = 2.58, p < 0.05]. 

GLM ANALYSES LOCALIZING THE dLPFC AND vmPFC ROIs 

A central goal of this study was to investigate the role of effec- 
tive connectivity between the area of vmPFC that has been widely 
associated with the computation of stimulus values at the time of 
choice, and an area of left dlPFC, in BA 46, that has been shown 
to exert a causal influence on discounting behavior in monetary 
intertemporal choices and implicated in self-control processes in 
various domains. (See the Introduction and Discussion for more 
details). In order to carry out the connectivity analyses, we first 
needed to localize these two brain regions in our sample. 

To do this, we first estimated a GLM of the BOLD responses that 
contained dSV and Accept (defined as 1 if the subject chose to accept 
the delayed option, and 0 otherwise) as parametric modulators at 
the time of decision (GLM-dSV). Based on previous studies, we 
expected that activity in vmPFC, PCC, and vStr would correlate 
with the dSV regressor, as these areas have been shown consistently 
to encode subjective values at the time of choice (Bartra et al., 2013; 
Clithero and Rangel, 2013). Note that in our task the immediate 
option was invariant ($25), whereas the delayed option changed 
every trial. Therefore, all trial-wise variation in the value of the 
delayed option is captured by dSV, even if the brain computes 
relative value signals (e.g., dSV — 25 or 25 - dSV). 

The contrast for the Accept regressor showed that, after con- 
trolling for dSV, a region of left dlPFC in BA 46 was more active 
when subjects chose the larger, delayed option (p < 0.05, SVC; 
Table 2). No regions were more active when declining the larger 
delayed reward in favor of the $25 today. Note that this increased 
activity when accepting delayed options is present after control- 
ling for dSV, indicating that it does not reflect a mere tendency 
of the subjects to choose larger rewards more frequently (indeed 
12 out of 27 subjects in our sample choose the objectively smaller 
reward today most often). 
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Table 2 | Regions more active when accepting delayed rewards 
controlling for discounted stimulus value in GLM-dSV. 

Region 



BA, Brodmann's Area. 

Height threshold t= 2.78 (p < 0.005) and extent of 20 voxels for table inclusion. 
Gray highlighting and * signify that the activation survives whole brain correction 
(p < 0. 05) for multiple comparisons at the cluster level. 

s ' Signifies that the activation survives small volume correction within an 
anatomical mask of vmPFC. 

s2 Signifies that the activation survives small volume correction within a 10 mm 
sphere centered on the estimated MNI coordinates from Figner et al. (2010) 
(x, y,z=-36, 30, 27). 

Peak voxel coordinates and cluster sizes within the small volume correction 
masks are listed in plain text below the corresponding clusters identified in the 
whole brain analysis. 



Table 3 | Regions reflecting discounted stimulus value at the time of 
choice in GLM-dSV. 



BA Side Cluster x y z Z 
size score 





Middle frontal gyrus 


6 


L 


1277 


-21 


9 


60 


4.56* 


Middle/inferior frontal gyrus 


46 


L 




-36 


36 


15 


4.06 


Putamen 




L 




-15 


6 


-3 


3.55 


Parietal lobe white matter 




R 


247 


30 


-33 


27 


4.18* 


Caudate 




R 




24 


-18 


24 


3.97 


Inferior temporal lobe 


19/37 


L 


61 


-48 


-57 


-6 


3.58 


Occipital cortex 


19 


L 




-48 


-63 


-12 


3.47 


Putamen 




R 


151 


9 


6 


-3 


3.26 


Thalamus 




L 




-3 


-18 


12 


3.21 


Thalamus 




L 




-12 


-6 


9 


3.1 


Middle frontal gyrus 


10/46 


R 


34 


42 


45 


21 


3.13 



BA, Brodmann's Area. 

Height threshold t= 2.78 (p < 0.005) and extent of 20 voxels for table inclusion. 
Gray highlighting and * signify that the activation survives whole brain correction 
(p < 0.05) for multiple comparisons at the cluster level. 

Labels in bold text are for the peak voxel within each cluster. Labels in plain text 
identify local maxima more than 8mm apart in different anatomical regions of 
larger clusters. 

The contrast for dSV revealed a large cluster (1277 voxels) with 
a peak in the middle frontal gyrus (Brodmann Area 6), extending 
into the putamen that was positively correlated with dSV (p < 
0.05 WBC; Table 3), but, to our surprise, no significant activity 
in the vmPFC region consistently linked to value computation at 
the group level (Bartra et al, 2013; Clithero and Rangel, 2013). In 
addition, none of the regions from the meta-analysis of negative 
correlations with subjective value in Bartra et al. (2013) showed 
decreasing activity as a function of dSV at whole brain or small 
volume corrected thresholds. Although there was no significant 
activity in canonical reward regions at the group level, inspec- 
tion of individual participant results revealed positive correlations 
with dSV within the vmPFC for many individuals. However, there 
was also a large fraction of participants who showed negative cor- 
relations with dSV in vmPFC, resulting in a summation of signed 
f-test coefficients that was close to zero at the group level. 

In order to examine the association between dSV and BOLD 
signals at the group level in a manner independent of the sign 
(+/— ) of this relationship, we re-estimated GLM-dSV, as well 
as a reduced version of this model excluding the dSV regres- 
sor, using Variational Bayes (see Materials and Methods). The 
logic of comparing the original version of GLM-dSV with the 
reduced model excluding dSV is that any differences in their 
fits to the data can be attributed to variance explained by dSV. 
Critically for our purposes, the amount of variance explained is 
unchanged by the sign of the regression coefficient allowing us 
to compute random effects statistics across individuals showing 
positive and negative effects of dSV. Following estimation of these 
models at the individual subject level, we compared their relative 
probabilities given the data from all subjects using BMS within 
a mask of reward value sensitive regions including vmPFC, vStr, 



BA Side Cluster x y z Z Region 
size score 



Cerebellum 




R 


74 


27 


-39 


-24 


4.56 


Occipital cortex 


30/18 


L 


1166 


-9 


-66 


12 


4.23* 


Inferior 


9 


L 


109 


-51 


3 


24 


4.17 


frontal/precentral 
















gyrus 
















Precentral/inferior 


6/9 


R 


118 


36 


-18 


36 


4.01 


frontal gyrus 
















Precuneus 


7 


R 


23 


24 


-60 


36 


3.92 


Anterior cingu- 


32/24/10 


L 


82 


-6 


30 


0 


3.85 


late/orbitofrontal 
















cortex 
















Small volume 






62 


-3 


30 


0 


3.59 s1 


corrected size/peak 
















Caudate/putamen 




L 


80 


-21 


24 


0 


3.85 


Thalamus 




R 


39 


6 


-6 


3 


3.61 


Cerebellum 




L 


38 


-54 


-54 


-30 


3.56 


Occipital cortex 


18 


R 


86 


21 


-87 


12 


3.54 


Middle/inferior 


46 


L 


38 


-30 


33 


15 


3.42 


frontal gyrus 
















Small volume 






15 


-33 


33 


18 


3.12 s2 


corrected peak 
















Thalamus 




R 


61 


21 


-27 


0 


3.38 


Middle frontal 


6 


R 


55 


30 


6 


54 


3.36 


gyrus 
















Cerebellum 




L 


91 


-36 


-57 


-27 


3.36 


Superior frontal 


6 


R 


33 


0 


-9 


72 


3.28 


gyrus 
















Precentral gyrus 


6 


L 


80 


-30 


-12 


60 


3.15 


Occipital cortex 


19 


R 


61 


36 


-78 


-12 


3.11 


Medial frontal 


6 


L 


23 


-9 


-6 


54 


3.08 


gyrus 
















Superior parietal 


7 


L 


79 


-18 


-75 


60 


3.08 


cortex/precuneus 
















Middle frontal 


10/9 


R 


26 


36 


48 


30 


3.03 


gyrus 
















Middle temporal 


37 


R 


29 


48 


-51 


0 


3.02 


gyrus 
















Superior frontal 


6 


L 


22 


-24 


6 


75 


2.98 


gyrus 
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FIGURE 2 | Voxels in meta-analytically defined reward value regions 
whose pattern of activity is better explained by a GLM including dSV 
as a regressor than a reduced version of the GLM omitting dSV. Voxels 
in violet are those within a mask of reward value sensitive regions including 
vmPFC, vStr, and PCC created from recent meta-analyses on reward value 
computation (Bartra et al., 2013; Clithero and Rangel, 2013) where the 
exceedance probability for GLM-dSV compared to the reduced version 
without dSV is 0.90 or higher. The exceedance probability was 0.90 or 
higher for the version of GLM-dSV including dSV in 83% of voxels within 
the meta-analysis conjunction. 



and PCC created from recent meta-analyses on reward value com- 
putation (Bartra et al, 2013; Clithero and Rangel, 2013). Here 
we compared the models based on their respective exceedance 
probabilities, which are measures that corresponds to the belief 
that a particular model is more likely than any other in the test 
set given the data from all participants. Figure 2 shows all vox- 
els in this mask where the exceedance probability for GLM-dSV 
compared to the reduced version without dSV is 0.90 or higher. 
Quantitatively, 83% of voxels in the meta-analysis conjunction 
had an exceedance probability of 0.90 or higher for the version of 
GLM-dSV including dSV, whereas only 4% had a value of 0.90 or 
higher for reduced GLM-dSV. This indicates that the dSV regres- 
sor explains a significant amount of the variance in the BOLD 
response in vmPFC, vStr, and PCC. 

Given that previous studies have found evidence consistent 
with the encoding of relative value signals in vmPFC at the time of 
choice (Boorman et al, 2009; Lim et al, 2011; Hunt et al., 2012), 
we carried out an additional generalized linear model (GLM- 
rdSV) . We hypothesized that there might be individual differences 
in the computation of the relative subjective value. In fact, a class 
of popular models in behavioral economics predicts that subjects 
will use as their reference item (i.e., the one that is subtracted 
when computing relative value) the option that they choose most 
frequently (Koszegi and Rabin, 2006). Based on this, we defined 
a relative discounted subjective value regressor (rdSV) that is 
given by dSV — 25 for those that picked the delayed option more 
than 50% of the time, and by 25 — dSV for those that select the 
immediate option most frequently. 

We estimated a new GLM (GLM-rdSV) with rdSV and Accept 
as parametric modulators at the time of decision. Consistent 
with the post-hoc hypothesis that value computations were made 



relative to the most frequent choice, we found that BOLD 
responses in vmPFC (p < 0.05 SVC, Figure 3A; Table 4) and the 
anterior superior temporal gyri (p < 0.05 WBC) were positively 
correlated with the modified value regressor. In addition, several 
regions including the anterior insula (Al), dorsomedial prefrontal 
cortex (dmPFC), inferior parietal cortex, middle frontal gyri, 
and posterior cingulate showed negative correlations with rdSV 
(p < 0.05 WBC; Figure 4A; Table 5). Both the vmPFC region 
that positively correlated with rdSV and the portions of the Al 
and dmPFC that negatively correlated with rdSV show consider- 
able overlap with results from recent meta-analyses (Bartra et al., 
2013; Clithero and Rangel, 2013) of reward value representation 
(Figures 3B, 4B). This overlap with previous results for both pos- 
itive and negative correlations with subjective value suggests that 
there is significant variation between subjects in how discounted 
subjective values are computed and that this computation may 
be related to choice frequencies (e.g., most often wait or rarely 
wait) consistent with our post-hoc hypothesis. We do not pre- 
sume or infer any causal relationships between choice frequency, 
and the directionality of relative discounted subjective value com- 
putations from these results, and it may be that a third as yet 
unknown variable drives choice preference, value computation, 
or both. However, this analysis provides us with a sample-specific 
ROI for vmPFC in which to test our main hypothesis about dlPFC 
modulation and the prediction of individual differences. 

Consistent with GLM-dSV, GLM-rdSV showed that regions of 
left dlPFC in BA 46 and 9 were more active when subjects chose 
the larger, delayed option (p < 0.05, SVC; Figure 5; Table 6). Just 
as in GLM-dSV, no regions were more active when declining 
the larger delayed reward in favor of the $25 today. Note that 
both GLM-dSV and GLM-rdSV control for the value of delayed 
rewards in a similar manner. The variance explained is the same 
in both models because in the individual subject GLMs only the 
sign of dSV regressor, and therefore, the sign on the regression 
coefficients changes while the explanatory power of the regressor 
remains the same. 

TESTS OF EFFECTIVE CONNECTIVITY 

Next, we used the ROIs in dlPFC-BA46 and vmPFC to test 
our first hypothesis; namely, that effective connectivity from left 
dlPFC-BA46 to vmPFC plays a critical role in delaying gratifica- 
tion. This test was carried out on time courses extracted from 
the vmPFC and dlPFC-BA46 ROIs identified in GLM-rdSV. We 
focused on the ROI in BA46 because a previous TMS study 
found a causal role for this region in choosing to wait for larger 
delayed rewards in monetary intertemporal choices (Figner et al., 
2010). Furthermore, our previous effective connectivity analy- 
ses of dietary self-control choices suggested other dlPFC regions 
active during self-control choices (e.g., BA9) might work through 
BA46 to modulate vmPFC (Hare et al, 2009). As explained in the 
Materials and Methods section, the test was performed in several 
steps. 

First, we estimated 4 different DCM families that were grouped 
based on how the experimental variables rdSV and Accept entered 
into the model as driving inputs (Figure 6A). Each family con- 
tained 16 models that varied on how the vmPFC and dlPFC-BA46 
affect each other as a function of three task events: fixation, choice 
periods, and choice periods when the delayed option is selected. 



www.frontiersin.org 



March 2014 | Volume 8 | Article 50 | 7 



Hare et al. 



Neural systems predicting individual discounting 




FIGURE 3 | Areas correlated with the rdSV regressor from GLM-rdSV. 

(A) A region of vmPFC showing increased activity as a function of rdSV 
(p < 0.05 SVC). (B) Voxels in vmPFC where the activation for rdSV in the 
current study overlaps with significant voxels in meta-analyses of positive 
correlations with subjective value by Bartra et al. (2013) and Clithero and 
Rangel (2013). All voxels shown in violet are significant in all three studies. 



Next, we used BMS to identify the most likely family of mod- 
els. We compared the models based on their respective exceedance 
probabilities, a measure of whether particular model is more 
likely than any other in the test set given the data from all partic- 
ipants. The most likely model family (exceedance probability = 
0.87), shown in Figure 6A, had two driving inputs (i.e., direct 
influences): an input to vmPFC given by the rdSV of the delayed 
option on every trial, and an input to dlPFC-BA46 given by 
Accept. 

Lastly, we examined the effective connectivity parameters 
between dlPFC-BA46 and vmPFC using BPA on the fully con- 
nected model (Figure 6B) with the most likely experimental 
inputs across subjects. We found increased signaling from dlPFC- 
BA46 to vmPFC at the time of choice relative to inter-trial 
fixation times, and further increases when subjects selected the 
later option (posterior probability > 0.90 and 0.95, respectively; 
Figure 6C). In contrast, the signaling in the other direction 
was not significantly different from zero. For completeness, we 
also compared the estimated DCM coefficients using one-sample 
f-tests, which lead to the same conclusion: effective connectivity 
parameters from dlPFC-BA46 to vmPFC increased during all 
choices and further increased when subjects selected the later 



Table 4 | Regions positively correlated with relative discounted 
stimulus value at the time of choice in GLM-rdSV. 



Region 


BA 


Side 


Cluster 
size 


X 


V 


z 


Z 
score 




Superior temporal 


21/22 


R 


944 


66 


-6 


-3 


4.11* 


gyrus 
















Cingulate gyrus 


24 


R 


152 


3 


-12 


42 


4.01 


Superior temporal 


21 


L 


641 


-39 


-9 


-9 


3.88* 


gyrus 
















Miiiciiur umyu 


39/9A/1D 


[_ 


I I o 


_Q 


30 


— O 


3 73 


late/orbitofrontal 
















cortex 
















Small volume 






81 


-9 


30 


-6 


3.73 s1 


corrected size/peak 
















Cingulate gyrus 


24/31 


L 


165 


-12 


48 


21 


3.7 


Precuneus 


3/5 


R 


96 


18 


-42 


45 


3.5 


Precuneus/cingulate 


5/7 


L 


40 


-9 


-42 


51 


3.36 


gyrus 
















Middle temporal 


39 


L 


32 


-51 


-72 


21 


3.08 


gyrus 

















BA, Brodmann's Area. 

Height threshold t= 2.78 (p < 0.005) and extent of 20 voxels for table inclusion. 
Gray highlighting and * signify that the activation survives whole brain correction 
(p < 0.05) for multiple comparisons at the cluster level. 

sl Signifies that the activation survives small volume correction within an 
anatomical mask of vmPFC. 

Peak voxel coordinates and cluster sizes within the small volume correction 
masks are listed in plain text below the corresponding clusters identified in the 
whole brain analysis. 

option [t(26) = 2.65 and f(26) = 3.80 respectively; p < 0.01], but 
signaling in the opposite direction did not increase significantly 
during any task events, suggesting that there is increased connec- 
tivity from dlPFC-BA46 to vmPFC during decisions to wait for 
larger delayed rewards, but not in the other direction. 

BETWEEN-SUBJECTS PREDICTION 

Next, we used the results of the DCM, to test our second hypoth- 
esis; namely, that it is possible to use inter-individual differences 
in the strength of effective connectivity between dlPFC-BA46 and 
vmPFC, as well as differences in local responses in those regions, 
to predict differences in discount rates. 

For each subject we estimated an elastic net regression model 
using only the data from the N— 1 other subjects, with discount 
rates as the dependent variable, and the estimated DCM parame- 
ters as the predictors. The estimated parameters of the model were 
then used to predict whether the discount rate of the excluded 
subject was above or below the mean of the group. The procedure 
was repeated to obtain a prediction for each subject. We found 
that the mean balanced accuracy (MBA) across all subjects was 
71% correct (95% posterior probability interval = 54 — 85%). 
In a complimentary analysis, we tested how well the continuous 
discount rate rankings (e.g., 1 = highest to 27 = lowest) esti- 
mated from the DCM parameters matched to those estimated 
from behavior, and found a significant correlation (Spearman's 
rho = 0.42, p < 0.02). Thus, the elastic net regression model can 
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FIGURE 4 | Areas negatively correlated with the rdSV regressor from 
GLM-rdSV. (A) Regions of the dmPFC and Al where activity decreased as 
a function of rdSV (p < 0.05 WBC). (B) Voxels in dmPFC and Al where 
responses to rdSV overlap with the meta-analyses results for regions that 
negatively correlated with subjective value at the time of choice in Bartra 
et al. (2013). All voxels shown in violet are significant in both studies. 



accurately predict both binary high low labels and the continuous 
ordinal rank of discount rates using DCM parameters. 

Next, we compared the accuracy of several versions of this 
prediction exercise, to test the specific role of the various 
components of the DCM in predicting the individual discount 
rates. Note that all parameters were estimated in the fully con- 
nected version of the model (shown in Figure 6) and were sim- 
ply omitted from the elastic net regressions during these tests. 
The logic of these tests is as follows: the prediction accuracy 
of a regression model that excludes a key parameter should 
drop, whereas excluding a parameter that does not play a role 
in intertemporal choice should not affect the model's abil- 
ity to predict the discount rates. The first test excluded the 
local driving input response parameters in vmPFC (Spearman's 
rho = 0.14, p = 0.48; MBA = 65%; 95% post. prob. int. = 
47 — 80%). The second test excluded the local driving input 
response parameters for dlPFC-BA46 (Spearman's rho = —0.03, 
p = 0.90; MBA = 60%; 95% post. prob. int. = 44 - 76%). The 
third test excluded the effective connectivity parameters from 
dlPFC-BA46 to vmPFC (Spearman's rho =-0.12, p = 0.54; 
MBA = 58%; 95% post. prob. int. = 41 - 74%). The fourth 



Table 5 | Regions negatively correlated with relative discounted 
stimulus value at the time of choice in GLM-rdSV. 



Region 


BA 


Side 


Cluster 
Size 


X 


V 


z 


Z 
score 




Inferior parietal lobe 


40 


R 


624 


42 


-57 


54 


4.96* 


Anterior 


13/45 


R 


153 


36 


30 


3 


4.85* 


insula/inferior frontal 
gyrus 
















Dorsomedial 


6/32 


L 


425 


-6 


21 


48 


4.84* 


prefrontal cortex 
















(SMA) 
















Superior 




R 




12 


27 


39 


4.84 


frontal/anterior 
















cingulate gyrus 
















Supplementary Motor 




R 




6 


18 


45 


4.84 


Area (SMA) 
















Anterior insula 


13 


L 


105 


-33 


21 


-3 


4.61* 


Inferior frontal gyrus 


45/47 


L 




-33 


21 


12 


4.61 


Middle frontal gyrus 


9 


L 


188 


-39 


12 


39 


4.55* 


Middle frontal gyrus 




L 




-45 


21 


12 


4.55 


Middle frontal gyrus 




L 




-51 


33 


36 


4.55 


Lingual 


30 


R 


66 


21 


-51 


6 


4.45* 


gyrus/posterior 
cingulate 
















Precuneus 


7 


L 


189 


-12 


-66 


36 


4.35* 


Inferior parietal lobe 


40/7 


L 


239 


-33 


-60 


45 


4.27* 


Middle frontal gyrus 


9/46 


R 


101 


51 


30 


39 


4.21* 


Middle frontal gyrus 




R 




48 


36 


27 


4.21 


Middle frontal gyrus 




R 




45 


18 


42 


4.21 


Thalamus 




L 


29 


-15 


-27 


12 


4.12 


Thalamus 




L 




-27 


-33 


12 


4.12 


Lateral orbitofrontal 


11 


R 


24 


30 


48 


-9 


4.06 


cortex 
















Thalamus 




R 


37 


9 


-12 


9 


3.97 


Pons/midbrain 




R 


26 


0 


-15 


-33 


3.6 



BA, Brodmann's Area. 

Height threshold t= 3.44 (p < 0.001) and extent of 20 voxels for table inclusion. 
A larger individual voxel threshold was used here to separate large clusters. 
Gray highlighting and * signify that the activation survives whole brain correction 
(p < 0. 05) for multiple comparisons at the cluster level. 

Labels in bold text are for the peak voxel within each cluster. Labels in plain text 
identify local maxima more than 8mm apart in different anatomical regions of 
larger clusters. 

test excluded the effective connectivity parameters in the other 
direction (Spearman's rho = 0.02, p = 0.91; MBA = 54%; 95% 
post. prob. int. = 37 - 70%). 

We found that omitting effective connectivity parameters 
between dlPFC-BA46 and vmPFC in either direction, or param- 
eters measuring local task induced responses within dlPFC-BA46 
or vmPFC reduced the accuracy to chance levels. Together, these 
findings show that the local responses in both areas, as well 
as both directions of effective connectivity between dlPFC and 
vmPFC, are critical for explaining the individual differences in 
discounting. Table 7 lists the relative size and direction of the 
effects of each DCM parameter on discount rates when estimating 
the model over all 27 participants. 
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FIGURE 5 | Increased activity in left dIPFC when choosing to accept 
larger, delayed rewards after controlling for subjective value (p < 0.05, 
SVC). The region of BA 46 shown here lies directly beneath the TMS 
stimulation site from Figner et al. (2010) that showed causal effects on 
temporal discounting behavior. 



Finally, we tested the specificity of these results with regard 
to the left dIPFC BA46 region. We replaced left dIPFC BA46 
in DCMs using the vmPFC with either the more posterior left 
dIPFC BA9 ROI that was also found to increase its activity when 
subjects chose the delayed rewards, or with an ROI created by mir- 
roring the 10 mm sphere centered on the estimated coordinates 
from Figner et al. (2010) to the right hemisphere. This resulted 
in two new DCMs and elastic net regression models. None of 
these combinations yielded significantly better than chance pre- 
dictions (best MBA = 55%) or significant correlations with the 
true discount rates. The results replacing left dlPFC-BA 46 with 
the analogous region in the right hemisphere are consistent with 
previous TMS results showing that only stimulation of the left 
hemisphere impacted choices for the delayed monetary rewards 
(Figner etal.,2010). 

DISCUSSION 

The results in this paper, in conjunction with previous reports 
(Hare et al, 2009, 201 la; Harris et al, 2013), suggest that a similar 
set of computational and neurobiological mechanisms are at work 
in tasks involving the delay of gratification in dietary and mone- 
tary intertemporal choices. In particular, we found that left dIPFC 
BA46 becomes more active in trials in which subjects choose the 
delayed option, which on average requires more self-control. We 
also found that the connectivity from left dIPFC BA-46 to a region 
of vmPFC widely associated with the computation of stimulus 
values (Bartra et al., 2013; Clithero and Rangel, 2013), increased 
at the time of choice, and especially during trials in which subjects 
chose to wait for the delayed reward. In addition, we were able 
to explain between-subject differences in discount rates using the 
estimated parameters from a DCM including the activity within 
dIPFC BA-46 and vmPFC, and the coupling between them, but 
only if the effective connectivity parameters between the two areas 
were included. 

These results parallel previous findings in the domain of 
dietary choice, in which individuals chose among foods that 
differed in their tastiness and healthiness (Hare et al., 2009, 
2011a; Harris et al., 2013). Although an explicit between-subject 
prediction exercise was not performed in those previous stud- 
ies, the data indicate a central role for dlPFC-vmPFC interactions 



Table 6 | Regions more active when accepting delayed rewards 
controlling for discounted stimulus value in GLM-rdSV. 



Region 


BA 


Side 


Cluster 


X 


V 


z 


Z 








Size 








score 


Cerebellum 




R 


65 


27 


-39 


-24 


4.55 


Inferior 


9/6 


L 


127 


-48 


3 


24 


4.27 


frontal/precentral 
















gyrus 
















Precentral/inferior 


6/9 


R 


205 


36 


-18 


36 


4.19* 


frontal gyrus 
















Occipital cortex 


30/18 


L 


1071 


-9 


-66 


12 


4.18* 


Anterior cingu- 


32/24/10 


L 


102 


-6 


30 


0 


3.99 


late/orbitofrontal 
















cortex 
















Small volume 






78 


-3 


30 


0 


3.69 


corrected peak 
















Caudate/putamen 




L 


77 


—21 


24 


0 


3.74 


Precuneus 


7 
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BA, Brodmann's Area. 

Height threshold t= 2.78 (p < 0.005) and extent of 20 voxels for table inclusion 
Gray highlighting and "signify that the activation survives whole brain correction 
(p < 0.05) for multiple comparisons at the cluster level. 

sl Signifies that the activation survives small volume correction within an 
anatomical mask of vmPFC. 

s2 Signifies that the activation survives small volume correction within a 10 mm 
sphere centered on the estimated MNI coordinates from Figner et al. (2010) 
(x, y,z = -36, 30, 27). 

Peak voxel coordinates and cluster sizes within the small volume correction 
masks are listed in plain text below the corresponding clusters identified in the 
whole brain analysis. 

in dietary self-control. This suggests that the mechanisms medi- 
ating self-control described in the Introduction are at work in 
both tasks, and thus helps to advance our understanding of com- 
mon computational and neurobiological components of various 
forms of self-control. In this model, vmPFC computes the value 
of options by identifying its various attributes, assigning value to 
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FIGURE 6 | Dynamic causal modeling results. (A) Schematic 
representations of the four DCM families compared in order to optimize 
the task related driving input to dIPFC BA 46 and vmPFC. Bayesian 
Model comparison showed that Family 1, outlined in black, was the most 
likely description of the data generating process. (B) Diagram of the fully 
connected model from the most likely family showing the posterior 
probabilities of coupling or coupling modulation greater than zero 
between vmPFC to dIPFC BA 46. Fixed refers to the baseline coupling 
during all time points. All Choices refers coupling modulation at the time 



of decision for all choices regardless of whether the immediate or 
delayed option was selected. Later Choices refers to coupling modulation 
during only those decisions when the larger, delayed option was chosen. 
(C) Bar chart showing the effective connectivity (EC) strengths in Hertz 
(Hz) between dIPFC BA 46 and vmPFC at different task periods. The 
colors and labels correspond to the diagram in panel (B). Asterisks 
indicate DCM parameters that are significantly different from zero when 
tested using both Bayesian parameter averaging (posterior probability > 
0.90) and one sample t-tests (p<0.01). 



them, and then integrating them into a net value for the option. 
A critical component of the model is that basic attributes (like 
immediate monetary payoffs, or the tastiness of foods) are pref- 
erentially incorporated into the values computed in vmPFC, but 
that more abstract attributes (like delayed monetary payoffs, or 
the healthiness of foods) are generally given less weight unless 
left dIPFC comes online and modulates activity in vmPFC, so 
that it weights all attributes according to the current goals (e.g., 
eat healthy or maximize monetary payoff). Note that the types 
of attributes that need to be represented and evaluated in both 
types of tasks are different, but that poor self-control could be 
attributed to the same source in both cases: reduced weighting of 
abstract attributes in vmPFC in the absence of dIPFC modulation. 

One limitation of the study must be emphasized. Our exper- 
iment is not able to differentiate between heterogeneity in the 
discount rates attributable to patience or self-control abilities 
(potentially mediated by differences in dIPFC functioning or 
connectivity), and heterogeneity due to differences in individual 
circumstances (e.g., immediate budgetary constraints) that are 
not directly associated with patience or self-control. Differences 
in individual circumstances, therefore, do not enter our predic- 
tion model and may be one reason why the model is less than 
perfectly accurate. In other words, our analysis cannot indicate 



if less patient subjects failed to wait for delayed rewards because 
they are unable to do so, or because their best option was to take 
the immediate monetary payout. 

These results provide novel interpretations of results in the siz- 
able literature on intertemporal choice paradigms. Consider three 
important examples. 

First, there has been a debate in the literature on whether or 
not there are multiple and competing value signals at work in 
self-control. In particular, previous findings (McClure et al., 2004, 
2007) have been interpreted as suggesting that vmPFC-VStr and 
dIPFC compute parallel but distinct value signals, with a vmPFC- 
VStr valuation system placing more value on immediate, concrete 
outcomes, and areas such as dIPFC computing the value of long- 
term, abstract goals. In this view, the quality of decisions depends 
on competition between the two valuation systems. In contrast, 
others have proposed that one value system integrates informa- 
tion about all stimulus attributes, both immediate and long-term, 
to form an overall value for the stimulus (Kable and Glimcher, 
2007, 2010). In this view, the quality of decision-making depends 
solely on the weighting of different stimulus attributes in value 
computation. The results here, and in previous work (Hare et al., 
2009, 2011a), suggest an obvious way of reconciling both views. 
In this class of tasks, choices seem to be driven by the stimulus 



www.frontiersin.org 



March 2014 | Volume 8 | Article 50 | 11 



Hare et al. 



Neural systems predicting individual discounting 



Table 7 | Regression coefficients predicting log(k) as a function of 
DCM parameters. 



Task period 


DCM parameter 


Elastic net parameter 


Fixed 


v — > d 


U.D 




d — > v 


1 Q 

— i.y 


All 

All choices 


v — > d 
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V SGlf 


n 1 




d — > v 


3.3 




dself 


-1.6 


Later choices 


v — > d 


0.8 




v self 


0.0 




d — > v 


-0.9 




dself 


0.2 


Driving inputs 


Value — > v 


0.4 




Accept — > d 


0.6 



This table reports the regression coefficients from the elastic net regression 
using the DCM parameters specified above to predict discount rates (log(k)). 
The regression is identical to those used in the prediction exercises described in 
the main text except that it was run with all 27 subjects at once. The regressions 
were estimated using the DCM parameters for both functional MRI runs sepa- 
rately and the values listed in this table represent the average of coefficients 
across runs for conciseness and clarity. 

Note that the DCM parameters have been z-scored across subjects so that com- 
paring the coefficients across parameters shows the relative size and direction 
of the effects of each DCM parameter on discount rates. See the main text of 
the Results section for quantification of the influence of each parameter on the 
model's ability to predict discount rates. Also, recall that smaller k values indi- 
cate lower discount rates and, therefore, parameters with negative regression 
coefficients increase the likelihood of choosing delayed reward options in our 
temporal discounting paradigm. 

The labels Fixed, All Choices, Later Choices, and Driving Inputs correspond the 

portions of the DCM described in the main text and shown in Figure 5. 

The label v — > d refers to signaling from vmPFC to dIPFC-BA 46. 

The label d — > v refers to signaling from dIPFC-BA 46 to vmPFC. 

The label v or d self refers to the parameters for the inhibitory self-connections 

to each region at the specified time points. 

The label Value — > v refers to an input equal to rdSV from GLM-rdSV into 
vmPFC. 

The label Accept — > d refers to an input equal the Accept regressor specifying 
later choices in GLM-rdSV. 

value signals encoded in a vmPFC-based valuation system, but the 
activation of dlPFC is critical for the deployment of self-control, 
because it appears to promote increased weighting of foresighted 
stimulus attributes in the vmPFC value signals as evidenced by 
increased effective connectivity to vmPFC during larger delayed 
choices. 

Second, our results provide a mechanistic explanation of the 
influential study of Figner et al. (2010), which found that apply- 
ing inhibitory TMS over left (but not right) dlPFC-BA46 resulted 
in a decrease in subjects' willingness to wait for delayed rewards 
(Figner et al, 2010). Consistent with the implication of a causal 
role for left dlPFC in self-control from these previous results, we 
find that this region is more active when subjects chose larger 
future rewards over payments on the same day, after control- 
ling for the subjective value of the payments. Furthermore, our 



data and analyses indicate that the left BA46 region of dlPFC 
contributes to delaying gratification by influencing the valuation 
process in vmPFC at the time of choice, rather than intervening 
after valuation has occurred, as was previously suggested in Figner 
et al. (2010). The previous suggestion by Figner and colleagues 
was based on their finding that choices over delayed options, 
but not the attractiveness ratings of those delayed rewards were 
affected by TMS to left dlPFC. However, our data on effective con- 
nectivity from dlPFC to vmPFC at the time of choice are more 
consistent with a mechanism in which dlPFC activity directly 
impacts valuation processes at the time of choice. We note, how- 
ever, that these results are not contrary to Figner and colleagues' 
assertion that the role of dlPFC is specific to decisions as opposed 
to outcome free ratings. 

Third, recent EEC and fMRI studies have found that individ- 
ual measurements of activity and connectivity within networks 
including left dlPFC taken at rest exhibited a sizable correlation 
with discount rates taken in separate behavioral tasks (Gianotti 
et al, 2012; Li et al., 2013). Similarly, a study of alcoholics found 
that responses in left dlPFC also correlated with behavior dur- 
ing intertemporal choices (Boettiger et al., 2007). Our results also 
provide a novel mechanistic explanation for these findings as a 
whole. Furthermore, our prediction exercises show that measures 
of effective connectivity between dlPFC and vmPFC are a critical 
aspect of being able to predict individual discount rates. 

We investigated the specificity of the dlPFC-vmPFC interac- 
tions in self-control by repeating a similar exercise replacing left 
dIPFC-BA 46 with left dIPFC-BA 9 or right dIPFC-BA 46. 

The specificity test using left dIPFC-BA 9 was motivated by the 
fact that this area was more active when subjects delayed gratifi- 
cation in previous dietary choice experiments (Hare et al., 2009, 
2011a) as well the current monetary choice dataset, although the 
activity did not survive whole brain correction in the current sam- 
ple. However, it did not result in significant correlations with or 
above average predictions of between-subject discount rates. This 
is consistent with our previous findings in dietary self-control 
where dlPFC-BA9 did not directly interact with vmPFC, but 
rather affected a more anterior region in BA46, near the region 
we find in the current intertemporal monetary choice task. 

The intertemporal choice task utilized here, as well as the 
dietary choice task that we have used in our previous related work 
(Hare et al., 2009), examines the deployment of self-control in 
the context of goal-directed choice. Other types of self-regulation 
might be better characterized by competition between habitual 
and goal-directed systems (Dayan et al., 2006; Balleine et al., 2008; 
Rangel et al, 2008), or by the type of response inhibition asso- 
ciated with action control in paradigms such as the go/no-go, 
Flanker, or Stroop tasks (Wager et al., 2005; Congdon et al., 2010). 
A critical question for future work is to systematically investigate 
the commonalities and differences between these various sources 
of self-regulation. 

Another avenue for further investigation is our finding that 
subjects appear to compute the discounted subjective value of 
delayed rewards relative to their most common choice, perhaps 
viewing this as a default. While not true in every case, the majority 
of subjects who most often chose the immediate reward appeared 
to positively encode a relative value signal in vmPFC equal to the 
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difference between the immediate reward and the larger delayed 
reward (i.e., $25 — dSV). They also showed negative correlations 
with this relative value signal in a network of regions that includes 
dmPFC, AI, and parietal regions consistently shown to negatively 
correlate with SV (Bartra et al., 2013). This network has been 
implicated in computations related to conflict, error process- 
ing, decision difficulty, and evidence accumulation (Carter et al., 
1998; Botvinick, 2007; Pochon et al, 2008; Venkatraman et al., 
2009; Wunderlich et al, 2009; Hare et al, 2011b). On the other 
hand, subjects who most often waited for the delayed reward fre- 
quently encoded the opposite relative value signal of dSV— $25 
in both sets of regions. This suggests that it is important to con- 
trol for reference point variation across subjects when examining 
the neural correlates of subjective values at the group level, but 
further investigation of this issue is clearly warranted. 

While our findings may at first seem contradictory to previ- 
ous reports where all subjects showed positive correlations with 
a value signal proportional to later reward — immediate reward, 
this can potentially be explained by important methodologi- 
cal differences. Many previous studies of intertemporal choice 
have customized the offer sets for each participant to main- 
tain an acceptance rate close to 50% for all subjects (Kable 
and Glimcher, 2007, 2010; Peters and Buchel, 2009, 2010). In 
contrast, we purposefully utilized the same offer set for all sub- 
jects to examine individual differences in neural responses. By 
keeping the response rate near 50% for all subjects, these pre- 
vious studies may have also generated a more homogeneous 
encoding of relative value in their participants avoiding the 
heterogeneity present in our dataset. These previous datasets 
also highlight that our findings with regard to relative value 
computations are likely driven by choice or action probabili- 
ties and not a function of discount rates or self-control ability 
because these previous datasets show that when subjects with 
high discount rates are presented with choices around their 
indifference points, they also have positive correlations with 
delayed reward values in vmPFC. Such changes in the direc- 
tionality of relative value computations as a function of choice 
or action probability represent an important target for future 
research. 

In summary, our data provide evidence that the dlPFC sup- 
ports the delay of gratification by modulating activity in a vmPFC 
region that reflects the stimulus value of available rewards. Our 
between-subjects prediction results indicate that both local activ- 
ity levels and connection strengths between these brain regions 
mediate delay of gratification tendencies in this task. These find- 
ings also suggest that examining effective connectivity parameters 
in pathological populations with self-control deficits may provide 
useful insights into the biological basis of their dysfunction. 
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